<!DOCTYPE html>
<html lang="en">

<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>GCC Testing Efforts</title>
<link rel="stylesheet" type="text/css" href="https://gcc.gnu.org/gcc.css" />
</head>

<body>

<h1>GCC Testing Efforts</h1>

<p>This page describes regular efforts to test GCC thoroughly, plus ideas
for additional testing.</p>

<p>For information about running the GCC testsuites, see
<a href="https://gcc.gnu.org/install/test.html">Installing GCC: Testing</a>.
For information about testsuite organization and adding new tests, see
<a href="https://gcc.gnu.org/onlinedocs/gccint/Testsuites.html">
Test Suites</a> in the GCC Internals manual and the README files in
the testsuite directories.</p>

<h2>Current efforts</h2>

<ul>
  <li>H.J. Lu has several automated Linux/ia32 and Linux/Intel64
      testers which build GCC and run the GCC testsuite as well as
      SPEC CPU 2K/2006 with various optimization options.  All
      test results are sent to the
      <a href="https://gcc.gnu.org/ml/gcc-testresults/">gcc-testresults</a>
      mailing list and any regressions are sent to the
      <a href="https://gcc.gnu.org/ml/gcc-regression/">gcc-regression</a>
      mailing list.</li>

  <li>Several people perform regular builds and regression test runs and
      send their test results to the
      <a href="https://gcc.gnu.org/ml/gcc-testresults/">gcc-testresults
      mailing list</a>.</li>

  <li>Jan-Benedict Glaw is runing a
      <a href="http://toolchain.lug-owl.de/buildbot/">build robot</a> that
      tries to build various cross-targets (stage1 only) on some machines.</li>
</ul>

<h2>Ideas for further testing</h2>

<ul>
  <li>Perform regular builds and testing of current GCC sources that
      are not already being reported regularly; see
      <a href="https://gcc.gnu.org/install/test.html">Installing GCC:
      Testing</a> for instructions on submitting test results.</li>
  <li>Build cross compilers and test with simulators as described in
      <a href="../simtest-howto.html">How to test GCC
      on a simulator</a>.</li>
  <li>If your system is beefy enough, do regular builds and test runs with
      RTL consistency checks enabled.  This slows the compiler down by an
      order of magnitude but has found plenty of bugs in the past.</li>
  <li>Set up an autobuilder to nag people in e-mail when they submit
      patches that break builds.  Ideally we would have one of these
      for at least each of the primary evaluation platforms listed in
      the current release criteria, but the more the merrier.
      Kaveh Ghazi &lt;<a href="mailto:ghazi@caip.rutgers.edu">
      ghazi@caip.rutgers.edu</a>&gt; suggests that the autobuilders
      should keep track of regressions in the number of warnings and nag
      patchers until the new warnings are fixed, just as for testsuite
      regressions.  It's important to have the autobuilders coordinate
      with each other, to avoid flooding contributors with mail.</li>
  <li>Build and test some or all of the following applications, some of
      which are part of the GCC release criteria.  Links for downloads
      and for information about the packages are available in the build
      and test guides.  If the instructions
      are incomplete for your target, update them.  Additions to this
      list (accompanied with build and test guides) are welcome.

<table class="border">
<tr><th>Name</th>
    <th>Language</th>
    <th>Build and test guide</th>
</tr>
<tr><td>Blitz++</td>
    <td>C++</td>
    <td><a href="testing-blitz.html">testing-blitz</a></td>
</tr>
<tr><td>Boost</td>
    <td>C++</td>
    <td><a href="testing-boost.html">testing-boost</a></td>
</tr>
<tr><td>FTensor</td>
    <td>C++</td>
    <td><a href="testing-ftensor.html">testing-ftensor</a></td>
</tr>
<tr><td>LAPACK</td>
    <td>Fortran 77</td>
    <td><a href="testing-lapack.html">testing-lapack</a></td>
</tr>
<tr><td>POOMA</td>
    <td>C++</td>
    <td><a href="testing-pooma.html">testing-pooma</a></td>
</tr>
<tr><td>Qt</td>
    <td>C++</td>
    <td><a href="testing-qt.html">testing-qt</a></td>
</tr>
</table>
  </li>

  <li>If the operating system kernel you use is normally compiled with
      GCC, try building it with the current sources.  Make sure it boots.
      If you're building with relatively stable GCC sources such as a
      release branch, use the newly built kernel for running further GCC
      tests (keeping in mind the NO WARRANTY section of the GPL).</li>
  <li>Build and test applications that are important to you; investigate
      and report any problems you find.</li>
  <li>Build and test packages that are normally available on your
      platform and for which you have access to source.</li>
  <li>Run benchmarks regularly and report performance regressions.</li>
  <li>Extend the
      <a href="http://toolchain.lug-owl.de/buildbot/">build robot</a> to also
      do local builds, run the testsuite, visualize test result differences
      and probably use something like
      <a href="http://buildbot.net/">BuildBot</a>. Some of the
      <a href="https://gcc.gnu.org/wiki/CompileFarm">Compile Farm</a> machines
      could also be used.</li>
</ul>

</body>

</html>
